Software applications are increasingly becoming critical in modern life. It is important to provide an adequate quality and minimize defects in software applications. Generally, in order to verify the software applications, software application may be tested before its execution. In order to test the software application, several methods have been proposed. Test automation is one of the methods used for software testing. Test automation is used to automate the execution of software tests and to compare actual outcome with predicted outcome of the software test results.
Test automation requires an automated test execution and results verification, termed as an automated test oracle. The automated test oracle is a reliable source used for expected outputs. The automated test oracle may be used to verify test case results that are executed on the software application under test. In order to verify behaviour of the software application, testers may require the test oracle for reliable expected behaviour of the software application. The automated test oracle may provide an output for a given input specified in the software application to verify the actual results generated by the software application.
In general, for automating the test oracle of reactive systems, a specification notation may be used to specify behavioural requirements for the reactive systems. In one example, the reactive system may include an automotive application. In one example, the automotive application may comprise sensors and actuators. The reactive systems may interact with their environment such as via sensors (inputs) and actuators (outputs). The behavioural requirements for the reactive systems are traditionally described using state-based or stream-based paradigms. In the state-based paradigm, the reaction of the automotive application to an input value is described based on current state of the automotive application. In the stream-based paradigm, the reaction of the automotive application is based on the pattern of stream of input values.
After specifying the behavioural requirements, creating test cases that cover the behavioural requirements is a challenging task and verifying the test results manually is tedious. In order to overcome issues presented above, automated test oracles may be used. However, when the automated test oracle is used, the systems/reactive systems require users to specify the behavioural requirements formally. Further, current specification notations are not user friendly while capturing the behavioural requirements and automated test oracles require specification of the behavioural requirements in formal specification notation.
Existing specification notations for automating the test oracles may include Statecharts, SCR, timing diagrams, and I/O Stream-based tables. Statecharts is a graphical state-based notation as disclosed in “Statecharts: A visual formalism for complex systems,” Science of Computer Programming, 1987 by D. Harel. The specification notations such as SCR are tabular notations, but are similar to the Statecharts. For sequence oriented specification requirements, the user may have to analyze and convert the specification requirements to state-based form. Further, timing diagram is a graphical and stream-based notation as disclosed in “A graphic language based on timing diagrams,” in Proceedings of the 13th Conference on Foundations of Software Technology and Theoretical Computer Science. London, UK, by C. Antoine, B. L. Goff, and J.-E. Pin. Although, timing diagram provides support for stream-based requirements, they are not suitable for state-based requirements. The I/O stream-based tables are stream-based requirements and are also tabular as disclosed in “Behavioural specification of reactive systems using stream-based I/O tables,” Software and Systems Modeling, 2011, by J. Thyssen and B. Hummel. However, the I/O Stream-based tables are not suitable for state-based requirements. Therefore, the above techniques result in the specifications that are not feasible to relate to the requirements of the system/reactive system, and the specifications are not relatable to the testers in viewing the requirements.